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Abstract 

A gap exists in the information needed to make intervention decisions with preschool children who are unresponsive to 
instructional intervention. Multi-Tiered System of Supports/Response to Intervention (MTSS/RTI) progress monitoring is helpful 
in indicating when an intervention change is needed but provides little information on what to change. Ecobehavioral 
observation data may provide this support through information on a child’s academic and other behaviors, given the 
opportunity to learn. We sought to investigate this hypothesis and develop benchmarks for decision making. Teachers (N = 
39) and two representative age-cohorts of preschool children (N = 117, 51% boys) were observed using an ecobehavioral, 
momentary time sample observation system (the Code for Interactive Recording of Children’s Learning Environments [CIRCLE]). 
Results provided insights into the content and amount of academic instruction children received, the responsiveness of 
children to instruction, and how context/teacher and child behaviors relationships were moderated by children’s level of 


Get Ready to Read (GRTR) literacy and Individualized Education Plan (IEP) status risk. Implications are discussed. 
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Early educators in their professional practice are expected 
to use data-based decision making when deciding which 
children may be at risk, and make changes in interventions 
based on data (Akers et al., 2015a, 2015b; Division for 
Early Childhood, 2014). While these concepts and practices 
are promoted in early intervention and early childhood spe- 
cial education (Division for Early Childhood, 2014), only 
recently have measures designed for use by practitioners 
engaged in universal screening and progress monitoring 
emerged in early childhood (Kaminski, Abbott, Bravo- 
Aguayo, Latimer, & Good, 2014; McConnell, Wackerle- 
Hollman, Roloff, & Rodriguez, 2014). These measures are 
very good at identifying children who are and are not mak- 
ing progress in the curriculum. However, because they are 
focused on learning outcomes, little information is avail- 
able about what the instructional problem conditions are, 
the most likely treatment adaptations needed, and how such 
decisions should vary given a child with a weak literacy 
skill level (literacy risk) and/or an Individualized Education 
Plan (IEP). The implications of this information gap cross- 
cuts nationally and locally reported trends in (a) low levels 
of instructional supports provided to children by preschool 


teachers in early education (Justice, Hamre, & Pianta, 2008) 
and inclusive early childhood special education classrooms 
(Carta & Driscoll, 2013; Guo, Sawyer, Justice, & Kaderavek, 
2013), and (b) the high rate of expulsion of children strug- 
gling to adhere to classroom routines (Gillam et al., 2008). 

Children who are not responding to intervention are not 
learning from the experiences arranged for them by the 
teacher. Several approaches exist that seek to identify instruc- 
tional problem conditions surrounding a child’s lack of 
response, and suggest change in instructional intervention, 
including functional behavioral assessment (FBA) and ecobe- 
havioral assessment (EBA; Watson, Gable, & Greenwood, 
2010). 

The goal of both approaches is to investigate response 
contingencies (environment—behavior interactions) that are 
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maintaining the problem and suggest potential solutions. 
Both consider children’s problems to be a function of proxi- 
mal environmental factors, and assessment is used to under- 
stand which set or combination of immediate environmental 
factors is likely maintaining the problem (Schwartz, 
Boulware, McBride, & Sandall, 2001). Once determined, 
the data also suggest a course of action for treatment or treat- 
ment hypotheses. These are then tested for effectiveness. 
And, both play a role in Multi-Tiered System of Supports/ 
Response to Intervention (MTSS/RTI) decision making. 

FBA (Hanley, Iwata, & McCord, 2003) is used primarily 
to inform children’s behavior problems and their manage- 
ment including likely reasons explaining the problem 
behavior and selection of interventions to ameliorate the 
problem. Notably, FBA has been extended to analyzing aca- 
demic performance problems in older students (Daly, Witt, 
Martens, & Dool, 1997). Standard reasons identified for 
lack of academic responding have included the following: 
Not enough time spent doing it (increase opportunity to 
learn/practice), Not enough help (provide more teacher 
prompting/feedback), Dont want to (strengthen motiva- 
tion), Done differently in the past (need to program general- 
ization), and Zoo hard (use explicit instruction to reduce 
difficulty). Each of these reasons suggest evidence-based 
procedures that are likely to overcome the problem and 
improve response in the context of instructional interven- 
tion, and they can be tested for function. FBA principles 
also have been extended from individuals to entire class- 
rooms to prevent behavior problems from emerging in the 
first place. For example, Classwide Function-Related 
Intervention Teams (CW-FIT; Kamps et al., 2015) is such 
an approach in early elementary classrooms that addresses 
most of the common functions of problem behavior. 

EBA (Morris & Midgley, 1990; Watson et al., 2010) also 
has a history of shedding light on children’s poor response 
to academic instruction in a range of school settings, includ- 
ing preschool (Carta, Sainato, & Greenwood, 1988; 
Greenwood, Abbott, Beecher, Atwater, & Petersen, 2016). 
Common reports from classroom settings have indicated 
infrequent exposures to academic content, low levels of 
children’s academic engagement, and increases in these 
outcomes given classwide instructional interventions such 
as Classwide Peer Tutoring (CWPT) and Literacy 3D 
(Greenwood et al., 2016; Greenwood, Delquadri, & Hall, 
1989). However, less is known about the use of EBA with 
individual cases of response to intervention. 

The standard reasons in understanding children’s aca- 
demic problems in EBA include (a) limited opportunity to 
learn defined by the low exposure to academic instruction 
and (b) limited academically engaged behaviors given 
instructional opportunities (Greenwood, Abbott, & Tapia, 
2003). Interventions for children who are not receiving suf- 
ficient opportunity to learn need to focus on boosting daily 


exposure to academic content by intentionally embedding 
such content into daily preschool activities. Given the 
opportunity to learn is provided, interventions for children 
who are not engaging in academic behaviors need better 
response opportunities, those more likely to evoke, rein- 
force, and maintain academic behaviors rather than compet- 
ing and/or other behaviors. Thus, all the FBA strategies 
previously mentioned also apply to EBA. 

The particular advantages of EBA (and the Code for 
Interactive Recording of Children’s Learning Environments 
[CIRCLE]) over FBA include direct assessment in authen- 
tic preschool contexts, measurement of instructional con- 
texts aligned with the language and literacy goals of 
preschool instruction and evidence-based practice, and stu- 
dent behavior given the opportunity to learn language and 
literacy skills. Thus, EBA tools like CIRCLE seem well 
suited to providing helpful information to individual cases 
of instructional failure in the MTSS/RTI decision-making 
cycle that address the questions of “Why is the problem 
happening?” “What should be done about it?” and “Is it 
being implemented?” (Tilly, 2008). 

Our long-term goal is to fill this information gap through a 
system involving (a) EBA information on instructional prob- 
lem conditions, (b) strong validity evidence that children’s 
behaviors are linked and sensitive to momentary variations in 
classroom instruction, (c) representative EBA benchmarks for 
comparative decision making, (d) a teacher coaching proce- 
dure for adapting instruction informed by an individual child’s 
EBA data relative to benchmarks, followed by (e) assessing 
individual effectiveness, consistent with MTSS/RTI decision 
making (Tilly, 2008) and the IEP process. 

The short-term goal of this report, however, was to 
investigate CIRCLE measurement and generate new evi- 
dence and benchmarks (Items a—c above). Thus, we exam- 
ined the following research questions in a cross-validation 
design using two time-displaced samples: 


Research Question 1: How was the opportunity to learn 
embedded in daily academic activity structures? This 
question sought to shed light on how teachers provided 
children with the opportunity to learn academic content 
and literacy focus given daily activity structures. 
Research Question 2: Was children’s academically 
engaged behavior significantly related to variations in 
children’s behavior during instruction? This question 
sought to determine the extent to which desired chil- 
dren’s behavior varied significantly with opportunities to 
learn provided during classroom instruction. 

Research Question 3: Did children’s personal charac- 
teristics moderate their instructional response dependen- 
cies? This question sought to shed light on whether or 
not personal characteristics (1.e., literacy risk and IEP 
status) moderated observed ecobehavioral relationships. 
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Method 


We explored these questions using a representative, extant 
sample of CIRCLE data (Greenwood et al., 2016), contain- 
ing a wealth of information on individual preschool chil- 
dren’s experiences with core academic instruction. Because 
teachers organize daily preschool experience by Activity 
Structures (D. R. Powell, Burchinal, File, & Kontos, 2008), 
we assessed the Academic Content and Teacher Literacy 
Focus embedded in daily activities along with children’s 
co-occurring behaviors (Academic Engagement, Other 
Engagement [OE], and Other Behaviors [OB]). 


Participants 


Programs. Preschool programs in two Midwestern school 
districts participated. The two programs were both half-day. 
District 1 offered parents a reverse inclusion program 
wherein children without IEPs were integrated with those 
with IEPs. Parents of children without IEPs paid tuition for 
them to attend the program, whereas Part B funds supported 
the services of children with IEPs. The average class size in 
the suburban district was 11, ranging from 8 to 14. The 
majority of children in these classrooms were White (81%). 
Forty percent of children had IEPs for speech and language 
services, developmental delay, behavior/emotional, occu- 
pational therapist/physical therapist, and autism. Only 10 
children (3%) were dual language. District 1 participated in 
both Years | and 2. District 2 participated in Year 2. District 
2 provided a combined state-funded Pre-K and Head Start 
program in an early childhood center. The rate of free or 
reduced-price lunch in the district was 60%. The average 
class size in this district was 13 children, ranging from eight 
to 15. The majority of children were African American 
(62%), 15% of children had IEPs, and four children (7%) 
were dual language. 


Teachers. In Year 1, 20 preschool teachers in District | par- 
ticipated. In Year 2, 19 teachers (13 teachers in District 1 
and six from District 2) participated. Eight District 1 teach- 
ers overlapped both years by participating again in Year 2 
(see Table 1). 


Children. Children (N = 117) from 39 Pre-K classrooms in 
the two school districts participated: 59 children in Cohort 
1 in Year 1 (52% boys) and 58 children in Cohort 2 in Year 
2 (48% boys) (see Table 1). All consented children’s early 
literacy skills were universally screened using the Get 
Ready to Read (GRTR) measure (Whitehurst & Lonigan, 
2001). 

To identify a representative subsample of target children 
to receive observations, we stratified by children’s literacy 
skill levels within each classroom. Three children in each 
classroom were randomly drawn, one each from the low, 


mid, and high GRTR standard score quartiles (= 100, SD 
= 15, quartile cuts were at +0.67 SD). Stratification was 
conducted in Cohort 1 in Year 2 and repeated again for 
Cohort 2 in Year 2. One of the three children in Class 6 had 
missing data in Year | (n = 2), and one additional child in 
Class 31 (n = 4) was advertently observed in Year 2 and 
included for a grand total of 117 children with complete 
data (see Table 2). All children in these samples were native 
English speakers. Twenty-seven percent of children had an 
IEP in Year 1, versus 28% in Year 2. There was a signifi- 
cantly greater concentration of children with an JEP in the 
lowest GRTR skills quartile group in Year 1—57%, 19%, 
and 10%; 7°(2) = 11.022, p = .004, in order by low, mid, and 
high quartile group—and in Year 2—43%, 29%, and 6%; 
1°(2) = 6.108, p = .047. 

Checks on the representativeness of the derived samples 
were made. First, the skew and kurtosis statistics reflecting 
the shape of the GRTR standard score distribution were 
roughly normal. Observed skewness overall was small and 
negative (—0.206) overall with a few more children scoring 
above rather than below the mean. Kurtosis indicated a 
degree of flatness (-0.917) where skew = 0 (tailness) and 
kurtosis (flatness) = 0 in the normal distribution. Somewhat 
similar distribution values were seen in each year, skew = 
—0.536 versus 0.154, kurtosis = —.579, vs. —1.09, respec- 
tively, between year cohorts. Second, the grand and class- 
room-level means and SDs in the majority of cases 
approximated the GRTR (M@ = 100 and SD = 15; see Table 
1). Third, tests of differences by classroom and year were 
not significantly different. Fourth, tests of divergence in 
GRTR quartile and IEP status groups produced significant 
mean differences as expected with no differences by year 
(see Table 1). 

We also examined whether or not these literacy risk 
groups were also divergent in Child Academic Engagement 
(CAE) and the Teacher Literacy Focus (TLF) they received. 
CAE quartile means were in the right direction at 28.2 (SD 
= 17.5), 30.4 (SD = 21.4), 32.1 (SD = 14.7) by low, mid, and 
high literacy risk groups (quartiles), respectively, but not 
significantly. Similar means for IEP status groups were 31.0 
(SD = 18.5) for the no-IEP group versus 28.2 (SD = 17.0) 
for the IEP group and also not significantly different. TLF 
quartile group means were 30.4 (SD = 21.6), 28.0 (SD = 
22.1), 31.0 (SD = 17.3) by low, mid, and high literacy risk 
groups (quartiles), respectively, and not significantly differ- 
ent. TLF means for IEP status were 29.9 (SD = 19.9) for the 
no-IEP group versus 29.3 (SD = 22.1) for the IEP group and 
not significantly different. 


Measurement 


EBA measurement. The CIRCLE (Version 2.0) is an ecobe- 
havioral observational measure of the alterable instructional 
processes that occur in classrooms including measures of 
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Table I. Sample Stratification on GRTR Early Literacy Skills Standard Scores. 


Year | (Cohort 1) Year 2 (Cohort 2) 
GRTR GRTR 
Variable N children M SD N children M SD 
Teachers/classes 
I 3 93.0 24.3 - - - 
2 3 101.3 14.6 - - - 
3 3 105.7 16.2 - - - 
4 3 95.7 18.8 - - - 
5 3 105.3 16.6 - - - 
6 2 94.5 29.0 3 96.7 21.6 
7 3 104.0 19.5 3 95.7 21.5 
8 3 103.7 19.3 3 98.7 16.3 
9 3 89.3 31.9 3 94.7 18.2 
10 3 103.0 13.0 3 100.0 12.5 
II 3 105.7 20.0 - - - 
12 3 100.0 17.5 3 96.0 19.7 
13 3 95.7 12.1 - - - 
14 3 99.0 25.0 - - - 
15 3 105.0 18.2 - - - 
16 3 99.0 19.7 3 101.3 17.1 
17 3 96.0 12.8 - - - 
18 3 97.3 20.6 - - - 
19 3 99.7 15.6 3 97.0 21.1 
20 3 98.0 24.3 - - - 
2I - = - 3 107.0 15.1 
22 - - - 3 96.0 13.7 
23 - - = 3 101.7 28. | 
24 - - = 3 98.3 18.9 
25 - - - 3 102.3 23.0 
26 - - - 3 101.7 21.1 
27 - - - 3 97.3 21.0 
28 - - - 3 88.7 11.0 
29 - - - 3 93.0 15.9 
30 - - - 3 92.0 10.5 
31 - - - 4 88.3 13.9 
Quartile groups* 
Low <89 18 78.1 8.3 2 80.1 5.6 
Mid >89, <I 10 2I 101.4 4.9 21 98.3 5.4 
High >110 20 117.2 3.8 16 117.6 5.2 
IEP groups? 
None 43 103.9 15.7 42 100.0 15.6 
IEP 16 88.1 14.6 16 89.3 14.5 
Overall 
Teachers 20 19 
Children 59 99.6 16.9 58 97.2 15.9 
Total teachers 39 
Total children 117 98.8 16.4 


Note. Count: 20 District | teachers Year | only, 8 District | teachers overlapped both years, | in Year 2. GRTR = Get Ready to Read; 
IEP = Individualized Education Plan; — = not participating. 
*Quartile main effect F(2, 111) = 4.287; p = 0001. “IEP main effect = F(2, 113) = 17.266, p = .0001. 
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Table 2. CIRCLE Descriptives by Year. 


Year | Year 2 

Construct Frequency Probability Frequency Probability Composites 
Academic Content 

|-Literacy 554 .23 634 27 Academic 

2-Numeracy 127 05 88 .04 Content 

3-Science/Social Studies 186 .08 90 .04 

4-Other 1,550 64 1,500 65 Other Content 
Activity Structures 

01-Center 1,061 44 769 33 Academic 

02-Story Time 157 06 207 09 Activities 

03-Large Group 738 31 712 31 

04-Small Group 308 13 109 05 

05-Individual Activity 7 .00 126 05 Other 

06-Eating 5 .00 175 .08 Activities 

07-Transition 134 06 207 .09 

08-Personal 3 .00 2 .00 

09-Therapy 0 .00 | .00 

10-Restricted 4 .00 4 .00 

| 1-None of Above 0 .00 0 .00 
Teacher Literacy Focus (Yes/No) 

|-Phonological Awareness 28 01 53 02 Yes 

2-Alphabet 91 .04 54 .02 

3-Story Comprehension 56 02 67 .03 

4-Other Comprehension 194 .08 95 .04 

5-Vocabulary 72 .03 36 02 

6-Reading 52 .02 92 .04 

7-Literacy 207 .09 312 13 

8-Other Focus 1,717 71 1,603 69 No 
Child Behavior 

02-Writing® 28 01 19 01 Academic 

03-Reading Aloud? 20 01 10 .00 Engagement 

04-Academic Manipulation? 69 .03 51 02 

05-Academic Verbal? 119 05 92 .04 

06-Academic Attention? 496 21 524 .23 

07-Play‘ 63 .03 61 .03 Other 

08-Sing, Perform Music‘ 36 01 49 02 Engagement 

09-Nonacademic Manipulation‘ 628 26 531 .23 

10-Gross Motor® 308 13 147 06 

| |-Eating® 19 01 124 .05 

12-Nonacademic Attention* 150 06 122 05 

13-None of Above 468 9 572 25 Other 

01-Competing Behavior® 13 01 10 .00 Behavior 
Total 2,417 1.00 2,312 1.00 


Note. CIRCLE = Code for Interactive Recording of Children’s Learning Environments. 
Composites are “Competing and Other Behaviors, "Academic, and ‘Other Engagement. 


children’s academic engagement (Greenwood et al., 2003; 
Greenwood & Kim, 2012; Watson et al., 2010). CIRCLE 
provides fine-grained data on young children’s engaged 
behavior given opportunities that are provided to learn lan- 
guage, literacy, numeracy, and science (Diamond, Justice, 
Siegler, & Snyder, 2013). CIRCLE is the outgrowth of a 


line of ecobehavioral observational research (Atwater, Lee, 
Montagna, Reynolds, & Tapia, 2009b; Greenwood & Kim, 
2012). CIRCLE is predicated on the fact that child behavior 
is malleable and responsive to situational variations pro- 
vided by teachers in the classroom environment (Fredricks, 
Blumenfeld, & Paris, 2004). 


224 


Topics in Early Childhood Special Education 37(4) 


CIRCLE provides a window on what individual children 
are doing, what the teacher is doing, and how the classroom 
is arranged during instruction, in the effort to promote 
desired academic outcomes through instruction (Neuman & 
Carta, 2011). For example, CIRCLE findings can report the 
likelihood of a child’s behavior in the context of (a) class- 
room Activity Structure (AS) (e.g., Centers, Story Time, 
Small Group), (b) Academic Content (AC) (e.g., Literacy, 
Numeracy, Science), and (c) TLF (e.g., Phonological 
Awareness, Vocabulary, etc.), among others. Thus, CIRCLE 
overcomes limitations of most preschool measures of class- 
room quality measures (Sabol, Soliday-Hong, Pianta, & 
Burchinal, 2013), and it is helpful in informing efforts to 
differentiate instructional interventions for children with 
and without disabilities (D. R. Powell et al., 2008). We used 
context and teacher and child constructs from the CIRCLE 
(see Table 2) to produce data on AC (five events, for exam- 
ple, Language, Numeracy), AS (11 events, for example, 
Centers, Large Group), and TLF (eight events, for example, 
Vocabulary, Alphabet). 


Children’s preliteracy screen. The GRTR measure was used 
(Phillips, Lonigan, & Wyatt, 2009; Whitehurst & Lonigan, 
2001). GRTR is a brief, 24-item screener that measures 
print knowledge, emergent writing, and phonological 
awareness with alpha coefficient (.78), reliability (.80), and 
validity (.58-.69) (Phillips et al., 2009). 


Child and family characteristics. Sociodemographic charac- 
teristics of the children and their families were assessed 
using a 25-item parent survey. For the child, date of birth, 
age, gender, race/ethnicity, disability status (IEP) were col- 
lected along with the primary caregiver’s educational attain- 
ment and the languages spoken in the home. 


Procedures 


This targeted sample was observed on multiple occasions in 
each of two years (Greenwood et al., 2016). For purposes of 
this investigation, however, only the first fall pretest occa- 
sion was used in each year as a baseline estimate. 
Observational data were collected during optimal times for 
implementation of academic activities according to teacher 
interview reports. Trained observers collected the data 
using tablet computers running data collection software 
(Atwater, Lee, Montagna, Reynolds, & Tapia, 2009a; 
Greenwood et al., 2012). Observations lasted for 90 min on 
the same day for about half (~180 minutes) of the program 
day experience—30 min per child for three focal children 
per classroom. 

To include representative contexts during 90 min, each 
child was observed for 10-min blocks. After observing 
one child for 10 min, the observer moved to the next child 
and then the third child. The observer then returned to the 


first child to start another block 20 min later. This 
produced a stratified sampling of events over the 90-min 
observation. 

During a live observation, observers’ recording of 
CIRCLE events was paced using momentary time sampling 
(Kennedy, 2005). Of the several time sampling methods, 
momentary time sampling is reportedly the best estimator 
of occurrence frequency (Ary, 1984; J. Powell, Martindale, 
& Kulp, 1975). In momentary time sampling, observers 
record events observed at the moment of an unobtrusive 
signal, with signals spaced at 15-s time intervals. CIRCLE 
data are recorded on an Android-based phone or tablet com- 
puter using a data entry program developed for this pur- 
pose. The software paced data entry by timing the intervals. 
The observer selected the event that best described the 
teacher’s behavior at the moment of the interval signal 
intervals from a drop-down entry screen. Context and child 
events were recorded in a similar manner. The sequence of 
Context, Teacher, and Child events was repeated until the 
end of the observation segment. 

Observers were trained by the developer of CIRCLE (the 
third author) and supervised by an observer coordinator 
(fifth author). Training included a combination of learning 
the CIRCLE taxonomy and associated event/behavior defi- 
nitions to 90% mastery, developing fluency using the 
CIRCLE software to enter observed classroom information, 
and certifying by passing three live interobserver reliability 
checks with the trainer at 90% overall agreement. Percentage 
of interobserver agreement was used as an indicator of reli- 
ability. An agreement was defined by an exact match 
between O1’s and O2’s record of occurrence in each 15-s 
momentary time sample. The percentage agreement was 
calculated as (100 [number of agreements / total intervals 
recorded in each observation]). Overall agreement averaged 
96.5% (range, 88%—100%) across all 50 paired checks. The 
mean agreement for the composites reported in this investi- 
gation was 98.3% (range, 75%-100%) for AC, 99.0% 
(range, 90%—100%) for AS, 96.6% (range, 75%—100%) for 
TLF, and 92.3% (range, 70%—100%) for CAE. There were 
no significant differences between these estimates in Year | 
(n = 26) versus Year 2 (n =24). 


Statistical Analysis 


The constructs in CIRCLE were measured, analyzed, and 
interpreted as manifest variables (see Table 2), those that 
can be observed, measured, and manipulated by teachers to 
influence learning outcomes (Yoder & Symons, 2010). To 
address research questions, we analyzed the data preserving 
co-occurrence as the focal variable (Kontos, Burchinal, 
Howes, Wisseh, & Galinsky, 2002). Individual children’s 
CIRCLE co-occurrence records consisted of the Context, 
Teacher, and Child behaviors/events in 45 s (three 15-s 
intervals). The combined Year | (2,417 records) and Year 2 
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(2,312 records) CIRCLE measurement occasions included 
a total of 4,729 momentary records. 

We consolidated the 13 child behaviors into three theo- 
retical composites including CAE, OE, and OB as shown in 
Table 2. CAE was the sum of writing, reading words or let- 
ters out loud, academic manipulation, academic verbal 
response, and academic attention frequencies. The OE com- 
posite was the sum of play, singing/music, nonacademic 
manipulation, gross motor, eating/drinking, and nonaca- 
demic attention. OB was a composite of inappropriate 
behaviors (e.g., aggression, noncompliance, etc.) and any 
other child behaviors not otherwise defined in CIRCLE. 
Consolidation also helped limit small cell sizes. 

Simple descriptive analyses (frequencies, mean/SD, pro- 
portions, and graphical displays) were used to characterize 
the sample in terms of child, teacher, and program charac- 
teristics. Univariate ANOVA based on the general linear 
model was used to test differences between classroom, 
GRTR quartile, IEP groups, and year cohorts, where score 
metrics were parametric. 

We used two-way marginal frequency tables for analyses 
of the CIRCLE data to address Questions 1 and 2. Tables 
contained frequency counts and probabilities (conditional 
and/or unconditional) of occurrence. To address Research 
Question 2, we also conducted chi-square tests of indepen- 
dence to examine differences in ecobehavioral relations. 
When significant, binary pairwise comparisons were made. 
Probabilities in Table 2 reflected the unconditional (base) 
occurrence of single variables as well as the conditional 
probability (co-occurrence) of events/behaviors (Tables 
3-5). The general stated form of the conditional probability 
was as follows, where the outcome (e.g., CAE) was pre- 
ceded by the expression given (e.g., TLF), the condition. 
For example, this conditional probability reflected the like- 
lihood of CAE occurring given that TLF occurred, or the 
probability (CAE/TLF). Conditional probabilities helped 
understanding of how teachers organized instruction and 
child response. 

Table 2 provides the CIRCLE raw descriptive frequen- 
cies and probabilities of occurrence and how they were 
combined to form composites used in subsequent analyses. 
Tables 3 and 4 describe the variability in children’s oppor- 
tunity to learn AC (see Table 3) and the TLF provided (see 
Table 4) given variation in the daily AS provided by the 
teacher. The last rows and columns in the tables provide the 
total count and unconditional probabilities. Table 5 simi- 
larly describes the probability of children’s behaviors (1.e., 
CAE, OE, and OB) co-occurring given AC, AS, or TLF. 
Table 6 provides the binary pairwise comparisons between 
these relationships. 

We used generalized linear mixed modeling (GLMM) to 
address Research Question 3. GLMM uniquely addressed 
features of the data that included lack of independence cre- 
ated by multiple observations per classroom (Kontos et al., 


2002) and sparseness in the occurrence of some events and 
behaviors. The approach also enabled modeling multiple 
sources of dependence due to child characteristics, without 
requiring that all events were experienced by all children to 
be included in the model. Additional advantages included 
the testing of main effects and interactions among predic- 
tors at any level of analysis, permitting the inclusion of 
incomplete child- or teacher-level data under the assump- 
tion of missing at random, and allowing for unbalance in the 
number of observations per sampling unit. The levels in the 
analysis were Child Records (NV = 4,729 at Level 1), Child 
(N = 117, intraclass correlation coefficient [ICC] = 0.17 at 
Level 2), and Classroom (N = 39, ICC = 0.06 at Level 3) 
using binary outcomes and with pairwise comparisons. 

We examined the extent that CAE was predicted by co- 
occurring AC (or AS or TLF) at Level 1, also varied system- 
atically by child literacy risk or IEP status. Literacy risk 
(GRTR, M = 98.4, SD = 16.4, range, 55-128) was centered 
at 100.0 in the models. IEP status was defined by either Yes 
(records = 1,289) or No (records = 3,440). We also included 
the effect of Year in the statistical models to examine repli- 
cation. Because of overlap in risk groups, a degree of col- 
linearity between the two moderators was observed; 
children without an IEP had significantly higher GRTR lit- 
eracy scores (M = 102.13, SD = 15.51) than children with 
TEPs (M = 88.37, SD = 14.32), t(2,490) = 28.77, p< .01,r, 
=—.50. Alpha values for pairwise tests were set to Bernoulli- 
corrected levels based on the number of comparisons. The 
outcome variable was CAE versus the composite of OE 
plus OB. The reference context in each model was the vari- 
able least associated with the occurrence of CAE as a basis 
for comparison with other variables. These reference con- 
texts were Other Content, Other Activities, and TLF = No. 


Results 


How Was the Opportunity to Learn Embedded 
in Daily Activity Structures? 


Teachers organized children’s preschool day by Activity 
Structures reflected in the Total columns of Tables 3 and 4 
for each year. The proportion of time that children spent in 
daily activities was Centers (.44), Large Groups (.31), Small 
Groups (.13), Other Activities (.06), and Story Time (.06) in 
Year |. This distribution was nearly identical in Year 2 (see 
Tables 3 and 4). 


Academic Content given Activity Structures. In general, chil- 
dren spent the majority of time in Centers not focused on 
learning academic content or literacy (see Table 3). In con- 
trast, children experienced the greatest opportunities to 
learn academic content and literacy during Story Time. 
Unfortunately, it occurred for the smallest amount of daily 
time compared with the other activities. The content 
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Table 4. Probability of Teacher Literacy Focus Given Activity Structures. 


Year | 


Year 2 


Teacher Literacy Focus 


Teacher Literacy Focus 


Activity Structures Indicator No Yes Total No Yes Total 
Center Prob (TLF/C) 89 ll 93 07 

Count 949 112 1,061 717 52 769 
Large Group Prob (TLF/LG) Jl Ad Jl 9 

Count 375 363 738 364 348 712 
Small Group Prob (TLF/SG) 74 26 69 31 

Count 227 8l 308 75 34 109 
Story Time Prob (TLF/ST) 13 87 .06 94 

Count 20 137 157 13 194 207 
Other Activities Prob (TLF/OA) 95 05 84 16 

Count 146 7 153 434 8l 515 
Total Prob (TLF) 7\ 29 1.00 69 31 1.00 

Count 1,717 700 2,417 1,603 709 2,312 


Note. Table data are sorted on Year | total occurrence of Activity Structures to assist visual inspection. TLF = Teacher Literacy Focus. 


children were most likely to experience during Year 1 was 
Other Content (p = .64, Year 1; p = .65, Year 2). The prob- 
ability of experiencing Literacy Content overall in both 
years was only .23 (year 1) and .27 (year 2), followed by 
Numeracy and/or Science/Social Studies, which ranged 
below .10 in both years. The vast majority of Literacy Con- 
tent exposure occurred during Story Time followed by 
Large and Small Groups particularly in Year 1 (see Table 
3). Children were mostly likely to experience Nonaca- 
demic Content not only during Other Activities in both 
years but also during Center, Large Group, and Small 
Group. 


Teacher Literacy Focus given Activity Structures. It was also the 
case that children experienced low levels of TLF, only .29 
in Year | and .31 in Year 2 (see Table 4). Children were 
most likely to experience TLF during Story Time activities 
(.87), followed by Large Group, Small Group, Center, and 
Other Activities in Year |. This pattern was largely the same 
in Year 2. 


Was Children’s Academic Engagement 
Significantly Associated With Variations in the 
Opportunity to Learn? 


Probability of CAE given Academic Content. The probability of 
CAE varied significantly given differences in Academic 
Content types (x7 = 1,691.4, df= 3, p< .01; see Tables 5 and 
6, upper panels). There was no significant difference by 
year. The overall probability of CAE during these opportu- 
nities to learn was only .30 in both years (see Table 5, Total 
rows). This compared with .50 for OE and .20 for OB, 
respectively. The probability of CAE was significantly 


more likely given that Literacy (.73), Numeracy (.72), or 
Science/Social Studies (.47) were taught, compared with 
Other Content (.10) (Bs = 3.29, 2.94, and 2.13, respectively, 
ps < .01; see Table 6). CAE was equally occurring and 
not significantly different between Literacy and Numer- 
acy, while Science/Social Studies co-occurred with CAE 
significantly less than with either Literacy or Numeracy 
(see Table 6). 


Probability of CAE given Activity Structures. The probability of 
CAE also varied significantly given differences in Activity 
Structures (y’ = 1,067.7, df= 4, p < .01; see Tables 5 and 6, 
middle panels). CAE was significantly more likely co- 
occurring during Story Time, Large and Small Groups, than 
Other Activities. CAE occurred infrequently in Center and 
Other Activities (see Table 5). These values replicated in 
Year 2. CAE was significantly more likely in Story Time, 
Large Groups, and Small Groups than Center Time (see 
Table 6). However, CAE was less likely to occur in Large 
and Small Groups than Story Time, and in Small Groups 
than in Large Groups. CAE was highest in Story (.83), and 
much lower in Large Group (.47), Small Group (.31), Cen- 
ter Time (.15), and the very lowest in Other Nonacademic 
Activities. 


Probability of CAE given Teacher Literacy Focus. The probabil- 
ity of CAE also varied significantly given differences in 
TLF (y’ = 1,461.5, df= 1, p<.01; see Tables 5 and 6, Lower 
Panels). CAE was significantly more likely to occur when 
the teacher was literacy focused than not-literacy focused 
(see Table 6). The occurrence of academic engagement was 
highest (.68) given TLF as compared with only .15 given 
No TLF (see Table 5). 
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Table 6. Pairwise Differences in the Probability of Child Academic Engagement Given Academic Content, Activity Structures, and 


Teacher Literacy Focus. 


Academic Content + Child Academic Engagement 


2 


x df P 
Chi-square test of independence 1,691.40 3 <.01 
Pairwise comparison B SE Z p Odds ratio 
Literacy vs. Other Content 3.29 0.11 31.18 <.01 26.96 
Numeracy vs. Other Content 2.94 0.18 16.15 <.01 18.85 
Science/Social vs. Other Content 2.13 0.16 12.95 <.01 8.44 
Numeracy vs. Literacy Content -0.36 0.19 -1.93 .20 0.70 
Science/Social vs. Literacy Content -1.16 0.17 -6.74 <.01 0.31 
Science/Social vs. Numeracy Content -0.80 0.22 -3.63 <.01 0.45 

Activity Structure — Child Academic Engagement 
a df p 
Chi-square test of independence 1,067.70 4 <.01 
Pairwise comparison B SE Z p Odds ratio 
Center vs. OtherActivities 0.45 0.18 2.53 .08 1.56 
StoryTime vs. OtherActivities 4.43 0.23 18.96 <.01 83.88 
LargeGroup vs. OtherActivities 2.59 0.17 15.31 <.01 13.33 
SmallGroup vs. OtherActivities 1.73 0.21 8.17 <.01 5.62 
StoryTime vs. Center 3.98 0.19 20.58 <.01 53.65 
LargeGroup vs. Center 2.14 0.11 19.87 <.01 8.52 
SmallGroup vs. Center 1.28 0.17 7.44 <.01 3.59 
LargeGroup vs. StoryTime -1.84 0.18 -10.15 <.01 0.16 
SmallGroup vs. StoryTime -2.70 0.23 -11.73 <.01 0.07 
SmallGroup vs. LargeGroup —0.86 0.16 5.32 <.01 0.42 
Literacy Focus > Child Academic Engagement 

ie df p 
Chi-square test of independence 1,461.5 | <.01 
Pairwise comparison B SE Z p Odds ratio 
Yes vs. No 2.92 0.09 31.18 <.01 18.56 
Note. Year was not significant in any analysis. 
Did Children’s Personal Characteristics Moderate Discussion 


Their Instructional Response Dependencies? 


IEP status proved to be the more active of the two moderators. 
Both significantly moderated the size of the CAE difference 
between Story Time and Other Activities. Children with 
greater literacy skills were significantly more likely to be aca- 
demically engaged (B = 0.04, p < .01), whereas children with 
an IEP were significantly less likely to be engaged (8 =—1.17, 
p=.02). IEP status also was observed to weaken other instruc- 
tion-engagement differences. Children with an IEP were sig- 
nificantly less likely to be academically engaged when 
Literacy versus Other Content was taught (8 =—0.81, p< .01), 
and when the teacher was versus was not literacy focused (6 = 
—0.54, p = .01). No other comparisons were significant. 


Closing the gap in the information needed to guide instruc- 
tional intervention decision making is a particular need for 
children who are not responsive to instructional interven- 
tion. Our purpose was to advance what we know about the 
potential use of EBA (the CIRCLE) in preschool MTSS/RT 
decision making by shedding light on “Why a child is not 
responsive to intervention?” “What should be done about 
it?” and “Is it being implemented?” 

To address these decision-making questions, informa- 
tion is needed on the number and sufficiency of the oppor- 
tunities to learn language and literacy in the preschool 
classroom. Results based on observations of 50% or more 
of a half-day session indicated that children in both years 


230 


Topics in Early Childhood Special Education 37(4) 


were not experiencing instruction that was particularly 
strong overall in terms of the opportunity to learn AC, or the 
TLF support teachers provided. Like previous reports, 
classroom teachers were providing a weak focus on aca- 
demic instruction (Carta & Driscoll, 2013; Greenwood et 
al., 2013; Guo et al., 2013; Justice et al., 2008; Neuman & 
Dwyer, 2009). The vast majority of time in these half-day 
classrooms was spent in Center Time where AC, TLF, and 
CAE were minimal. Teachers also provided activities for 
children that were mostly whole group and not known to 
promote high CAE. Small Group and Individual Activities 
known to promote CAE in more intensive MTSS/RTI inter- 
ventions (Tiers 2 and 3) were used infrequently by teachers 
or, in the case of Individual Activities, rarely at all (see 
Table 2). These findings were consistent with other reports 
indicating a preschool teacher’s preferences for providing 
instruction in Large Groups and Story Time compared with 
Small Group or Individual settings (D. R. Powell et al., 
2008). 

Data such as these are important to MTSS/RTI decision 
making in a number of ways. One is evaluation of individ- 
ual children not making progress on screening and progress 
monitoring measures who may benefit from greater oppor- 
tunity to learn and teacher support. For example, the teacher 
could arrange a Tier 2 or 3 experience for a child to add 
more content intensity to instruction received in Tier 1. 
Another is an effort to strengthen the Tier 1 core literacy 
program. Yet another is providing professional develop- 
ment to include an intentional content emphasis to increase 
children’s opportunity to learn literacy skills. And, one 
more is heightened focus on evidence-based strategies. For 
example, decision makers might well consider introducing 
evidence-based strategies and procedures known to increase 
teacher—student interactions and child engagement with 
academic content (Greenwood et al., 2016). For example, 
embedding content instruction during Center and Large 
Group activities (Spencer, Goldstein, & Kaminski, 2012; 
Woods, Kashinath, & Goldstein, 2004) and making instruc- 
tion more explicit and interactive (August & Shanahan, 
2006; Goldenberg, 2008) help provide greater support. 

To address these decision-making questions, informa- 
tion also is needed on the extent that CAE is frequently 
occurring given the opportunities and supports provided by 
the teacher. Results indicated that CAE covaried widely 
given differences in AC, AS, and TLF. CAE was signifi- 
cantly more likely to occur during Literacy, Numeracy, and 
Science/Social Studies content instruction and least likely 
to occur in Nonacademic Contexts. Conversely, children’s 
other engaged behaviors were significantly more likely 
during Nonacademic, Other Activities, and No TLF. 

CAE was occurring mostly in Story and Large Group 
activities but not in Centers, Small Group, Individual, or 
Other Activities. These findings were replicated in a second 
year strengthening the evidence that what teachers did and 


how they arranged the classroom activity systems actually 
made a difference in the occurrence of academic behavior 
of children. These results add to our confidence in the infer- 
ences that can be made based on CIRCLE data. The data 
serve as a starting point for the development of comparative 
benchmarks in a representative sample for use in individual 
decision making. For example, CIRCLE informs the ques- 
tion, “What is the problem?” by providing the evidence 
needed to rule in or out low opportunity and academic 
engagement as the problem. CIRCLE data informs the 
question, “What should be done?” by suggesting proce- 
dures and strategies most likely to address the problem. 
Last, CIRCLE informs the question, “Is it being imple- 
mented?” by supplying evidence that planned changes have 
been made and are increasing opportunity to learn and aca- 
demic engagement. 

To address decision-making questions, information also 
is needed regarding whether or not children’s literacy risk 
and IEP status moderate response to instruction. Findings 
indicated moderation in several cases. Children with an IEP 
risk moderated three of the eight possible comparisons, 
while higher literacy risk moderated only one of these. Both 
risks adversely affected the relationship of CAE given the 
Story Time Activity. Children with IEPs additionally had 
significantly weaker CAE relationships given Literacy 
Content and TLF = yes. These findings were particularly 
relevant to identifying instructional problems and also tar- 
geting intervention strategies likely to provide additional 
supports where higher levels of attention and academic 
responding are needed. Implications were that children with 
IEPs in particular need more intensive content-focused 
instruction than do typically developing children that is lon- 
ger in duration and greater in frequency. 


Strengths of the Study 


This research produced detailed information on how pre- 
schools, teachers, and children actually operated in authen- 
tic settings in a sample of children with representative 
literacy skills attending half-day programs. Initial results 
indicated no measurable differences in CAE or in TLF sup- 
port between risk groups divergent in GRTR literacy skills 
even though their instructional needs were different. Most 
of the field’s current information in this area is based on 
results from aggregated daily observations, rather than the 
rich detail provided by momentary co-occurrence data 
within observations. These data confirmed that teachers 
were not differentiating children’s experiences during the 
time observed. 

It was also a strength that the momentary results revealed 
typically unseen variations in classroom ecobehavioral con- 
structs helpful in addressing MTSS/RTI decision making. 
Findings provided insights into the content and amount of 
academic instruction children received, the responsiveness of 
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children to instruction, and how these teacher and child behav- 
iors were moderated by children’s level of GRTR literacy and 
IEP status risk. These were important findings because they 
provided new knowledge that the momentary instructional 
interactions that teachers arranged for these children appeared 
to be less effective than with children not at risk. Moreover, 
the consistency of results across 2 years was notable and 
indicative of the representativeness of these teaching practices 
and their impact of children’s engagement. 


Limitations 


We only used the first observation occasion in this extant 
sample in the beginning (fall) of each year. One observation 
per year, however, one is not likely representative of an 
entire preschool year. Yet, the degree of stability seen in 
these estimates at the beginning of the year, one year to the 
next, suggested uniformity in the system of instruction pro- 
vided by teachers in both years, and its relationship to chil- 
dren’s response to instruction. However, because a majority 
of the data was collected in the same school district with 
some teachers repeating in both years, even though student 
cohorts were different, this was a competing explanation. 

The instruction-response dependencies operationalized 
in this research were temporal correlations and were not 
causal estimates of ecobehavioral relationships. The condi- 
tional probabilities reflected the temporal association (lag = 
0, probability of occurrence) within a 45-s time window 
(three consecutive 15-s intervals of ecology, teacher, and 
child recording). This was the best level of precision possi- 
ble given the momentary time sample procedures used in 
CIRCLE. Clearly, a greater level of temporal precision 
would be desired; however, reliable use by observers is 
always a psychometric and practical limitation given the 
complexity reflected in the taxonomy of events to be moni- 
tored and recorded. Given the large number of events/ 
behaviors reliably coded in CIRCLE, the challenge in using 
a more frequent, real-time sampling method remains to be 
overcome (Dorsey, Nelson, & Hayes, 1986). 

A degree of collinearity between our two moderators 
was observed, likely due to the low literacy skills shared by 
both groups. Children without an IEP had significantly 
higher GRTR literacy scores compared with children with 
IEP status. This overlap centered on the children’s weaker 
language, literacy, and cognitive skills measured by the 
GRTR. Thus, additional contributions of developmental 
delay or disability to this issue are not known. While low 
literacy risk and IEP status were included as moderators in 
the current study, it was not possible to include children 
with dual-language learner (DLL) status because they 
were not sufficiently represented in this extant sample. 
Thus, this widely known moderator of children’s response 
to instructional intervention relationship has yet to be inves- 
tigated using CIRCLE. 


Future Research and Directions 


Future research could profitably examine these questions in 
different districts with different curricula and early literacy 
goals, while also sampling additional child characteristics 
(e.g., DLLs, etc.). Based on these findings, CIRCLE obser- 
vation data appeared to fill the previously mentioned gaps 
between progress monitoring indication and treatment for- 
mulation/implementation for children not responding to an 
instruction intervention. However, more research and dem- 
onstration is warranted. A next step could be to employ 
CIRCLE data in the MTSS/RTI decision-making cycle for 
indicated individuals. These can be single case and/or group 
experimental (causal) design studies focused on demon- 
strating that intervention changes made given CIRCLE 
information actually lead to better response and learning 
outcomes. More work also is needed to replicate the current 
descriptive CIRCLE findings in other samples of programs 
and children, with more frequent assessment occasions, dif- 
ferent years adding incrementally to benchmark estimates. 
Such cross-validating work will extend the generality of the 
ecobehavioral observed relationships between instruction 
and children’s response. Work also is needed to untangle 
child moderators when they share a common construct, for 
example, low literacy English skills, as do children with 
IEPs and who are DLL. 


Implications for Practice 


A primary implication of this investigation is that preschool 
teachers need to devote more time and precision to their aca- 
demic instruction and literacy support, and they need to do so 
differentially to ensure that children with higher risk status 
are recelving more intensive instruction. It is reasonable to 
assume with the right supports that classroom teachers can 
raise classroom-level averages by intentionally embedding 
the use of evidence-based instructional practices in daily 
schedule of activities. Such support may well come from 
CIRCLE data in terms of making changes to core instruction 
with a greater likelihood of benefiting an individual child as 
well as all the children in the class and program. 


Conclusion 


These results add to our knowledge of how preschool 
instructional factors are associated with children’s engage- 
ment in learning academic content in inclusive classrooms, 
and how that information may help solve cases of response 
to intervention. This lens that the CIRCLE provides on chil- 
dren’s experiences has implications for planning individual- 
ization of specific children (including the IEP process), 
addressing the lack of response to instructional interven- 
tion, and evaluating the strength of the core instruction 
received by all children. The success of this proposed use of 


232 


Topics in Early Childhood Special Education 37(4) 


CIRCLE remains to be demonstrated in future research and 
practice. 
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